New architecture GPUs like A100 are now equipped with multi-instance GPU (MIG) technology, which allows the GPU to be partitioned into multiple small, isolated instances. This technology provides more flexibility for users to support both deep learning training and inference workloads, but efficiently utilizing it can still be challenging. The vision of this paper is to provide a more comprehensive and practical benchmark study for MIG in order to eliminate the need for tedious manual benchmarking and tuning efforts. To achieve this vision, the paper presents MIGPerf, an open-source tool that streamlines the benchmark study for MIG. Using MIGPerf, the authors conduct a series of experiments, including deep learning training and inference characterization on MIG, GPU sharing characterization, and framework compatibility with MIG. The results of these experiments provide new insights and guidance for users to effectively employ MIG, and lay the foundation for further research on the orchestration of hybrid training and inference workloads on MIGs. The code and results are released on https://github.com/MLSysOps/MIGProfiler. This work is still in progress and more results will be published soon.
translated by 谷歌翻译
The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.
translated by 谷歌翻译
We present a neural flow wavefunction, Gauge-Fermion FlowNet, and use it to simulate 2+1D lattice compact quantum electrodynamics with finite density dynamical fermions. The gauge field is represented by a neural network which parameterizes a discretized flow-based transformation of the amplitude while the fermionic sign structure is represented by a neural net backflow. This approach directly represents the $U(1)$ degree of freedom without any truncation, obeys Guass's law by construction, samples autoregressively avoiding any equilibration time, and variationally simulates Gauge-Fermion systems with sign problems accurately. In this model, we investigate confinement and string breaking phenomena in different fermion density and hopping regimes. We study the phase transition from the charge crystal phase to the vacuum phase at zero density, and observe the phase seperation and the net charge penetration blocking effect under magnetic interaction at finite density. In addition, we investigate a magnetic phase transition due to the competition effect between the kinetic energy of fermions and the magnetic energy of the gauge field. With our method, we further note potential differences on the order of the phase transitions between a continuous $U(1)$ system and one with finite truncation. Our state-of-the-art neural network approach opens up new possibilities to study different gauge theories coupled to dynamical matter in higher dimensions.
translated by 谷歌翻译
Gauge Theory plays a crucial role in many areas in science, including high energy physics, condensed matter physics and quantum information science. In quantum simulations of lattice gauge theory, an important step is to construct a wave function that obeys gauge symmetry. In this paper, we have developed gauge equivariant neural network wave function techniques for simulating continuous-variable quantum lattice gauge theories in the Hamiltonian formulation. We have applied the gauge equivariant neural network approach to find the ground state of 2+1-dimensional lattice gauge theory with U(1) gauge group using variational Monte Carlo. We have benchmarked our approach against the state-of-the-art complex Gaussian wave functions, demonstrating improved performance in the strong coupling regime and comparable results in the weak coupling regime.
translated by 谷歌翻译
从搜索效率中受益,可区分的神经体系结构搜索(NAS)已发展为自动设计竞争性深神经网络(DNNS)的最主要替代品。我们注意到,必须在现实世界中严格的性能限制下执行DNN,例如,自动驾驶汽车的运行时间延迟。但是,要获得符合给定性能限制的体系结构,先前的硬件可区分的NAS方法必须重复多次搜索运行,以通过反复试验和错误手动调整超参数,因此总设计成本会成比例地增加。为了解决这个问题,我们引入了一个轻巧的硬件可区分的NAS框架,称为lightnas,努力找到所需的架构,通过一次性搜索来满足各种性能约束(即,\ \ suesperline {\ textIt {您只搜索一次}})) 。进行了广泛的实验,以显示LINDNA的优越性,而不是先前的最新方法。
translated by 谷歌翻译
线云虽然在先前的工作中受到评价不足,但与从多视图图像中提取的点云相比,可能对建筑物的结构信息进行了更紧凑的结构信息。在这项工作中,我们建议第一个处理用于构建线框抽象的线云的网络。该网络将线云作为输入,即从多视图图像提取的3D线段的非结构和无序集,并输出基础建筑物的3D线框,该建筑物由稀疏的3D连接组组成,由线段连接, 。我们观察到一个线斑块,即一组相邻的线段,编码足够的轮廓信息,以预测潜在连接的存在甚至3D位置,以及两个查询连接之间的连通性的可能性。因此,我们引入了两层线斑变压器,以从采样线贴片中提取连接和连接性,以形成3D构建线框模型。我们还介绍了带有地面3D线框的多视图图像的合成数据集。我们广泛证明,在多个基线建筑重建方法上,我们的重建3D线框模型可显着改善。
translated by 谷歌翻译
建模用户从历史行为中的动态偏好在于现代推荐系统的核心。由于用户兴趣的多样性,最近的进步建议多功能网络将历史行为编码为多个兴趣向量。在实际情况下,通常会一起检索相应的捕获兴趣项目,以获取曝光并收集到培训数据中,从而产生兴趣之间的依赖性。不幸的是,多息网络可能错误地集中在被捕获的利益之间的微妙依赖性上。被这些依赖性误导了,捕获了无关的利益和目标之间的虚假相关性,从而导致训练和测试分布不匹配时预测结果不稳定。在本文中,我们介绍了广泛使用的Hilbert-Schmidt独立标准(HSIC)来衡量被捕获的利益之间的独立性程度,并经验表明,HSIC的持续增加可能会损害模型性能。基于此,我们提出了一个新颖的多息网络,称为深稳定的多功能学习(Desmil),该网络试图通过学习权重以训练样本的学习权重消除捕获的兴趣中微妙的依赖性的影响因果关系。我们对公共建议数据集,大规模工业数据集和合成数据集进行了广泛的实验,这些数据集模拟了分布数据的数据集。实验结果表明,我们提出的Desmil的表现优于最先进的模型。此外,我们还进行了全面的模型分析,以揭示Desmil在一定程度上工作的原因。
translated by 谷歌翻译
多视图学习通过LEVERAG-ING-ING-ING相同对象之间的关系来完成分类的任务目标。大多数现有方法通常关注多个视图之间的一致性和互补性。但并非所有这些信息都非常有用于分类任务。相反,它是扮演重要作用的具体辨别信息。钟张等。通过联合非负矩阵分组探讨不同视图中的共同视图中存在的判别和非歧视信息。在本文中,我们通过使用跨熵损耗函数来改善该算法来改善目标函数更好。最后,我们在相同数据集上的原始实施更好的分类效果,并在许多最先进的算法上显示其优越性。
translated by 谷歌翻译
我们研究神经网络量子状态的无限限制($ \ idty $ -nnqs),它通过集合统计表现出代表性,以及易衰减的梯度下降动态。根据神经网络相关器表示瑞尼熵的集合平均值,并提出了表现出体积法纠缠的架构。开发了一种用于研究神经网络量子状态(NNQS)的梯度下降动态的一般框架,使用量子状态神经切线内核(QS-NTK)。对于$ \ infty $ -nnqs,简化了训练动态,因为QS-NTK变为确定性和常数。导出分析解决方案用于量子州监督学习,允许$ \ infty $ -nnqs恢复任何目标波段。横向场介绍模型有限和无限NNQ的数值实验和Fermi Hubbard模型表现出与理论的优秀协议。 $ \ infty $ -nnqs开辟了研究其他物理应用中的纠缠和培训动态的新机会,例如在寻找基地。
translated by 谷歌翻译
尽管概念化已经在语义和知识表示中进行了广泛研究,但找到最准确的概念短语来表征在快速增长的社交媒体上表征文本片段的主要思想仍然具有挑战性。这部分归因于以下事实:大多数知识库都包含世界的一般术语,例如树木和汽车,它们没有定义的力量或对社交媒体应用程序用户不够有趣。另一个原因是,自然语言的复杂性允许使用时态,否定和语法改变语言的逻辑或重点,从而传达了完全不同的含义。在本文中,我们提出了标签,这是一个高质量的概念匹配的数据集,该数据集由10,000个标记的精细概念和网络风格的自然语言句子组成,并从开放域社交媒体中挖出。我们考虑的概念代表了在线用户的趋势兴趣。与标签相关的是这些细粒度概念和实体的概念图,以提供结构上下文信息。我们在标签上评估了广泛的流行神经文本匹配模型以及预先训练的语言模型,并指出他们以最合适的概念标记社交媒体内容的不足。我们进一步提出了一种新颖的图形匹配方法,该方法通过更好地利用概念图中的结构上下文和句子中语义单元之间的逻辑相互作用在句子中通过句法依赖性解析来展示出色的抽象和概括性能。我们开源标签数据集和提出进一步研究的建议方法。
translated by 谷歌翻译